Information processing system, customer identification apparatus, and information processing method

ABSTRACT

According to the present invention, there is provided an information processing system ( 100 ) including an acquisition unit ( 11 ) that acquires an image, which is generated by a camera, of a face, a selection unit ( 14 ) that extracts a person in the acquired image, and selects a person who performs a settlement process based on at least one of a distance from a checkout apparatus, an appearance frequency, and an appearance period of time in a case where a plurality of persons are extracted, an identification unit ( 12 ) that identifies the selected person based on the image in which the person is included, and database information in which a feature value of a face is registered, a settlement unit (checkout unit ( 21 )) that performs the settlement process with respect to the identified person.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 17/233,920 filed on Apr. 19, 2021, which is acontinuation application of U.S. patent application Ser. No. 16/466,809filed on Jun. 5, 2019, which is issued as U.S. Pat. No. 11,017,400,which is a National Stage Entry of international applicationPCT/JP2017/032080 filed on Sep. 6, 2017, which claims the benefit ofpriority from Japanese Patent Application 2016-248811 filed on Dec. 22,2016, the disclosures of all of which are incorporated in their entiretyby reference herein.

TECHNICAL FIELD

The present invention relates to an information processing system, acustomer identification apparatus, an information processing method, anda program.

BACKGROUND ART

Patent Document 1 discloses an information processing system using aface authentication process for a settlement operation. Patent Document1 discloses that the information processing system acquires informationwhich includes face information of a visitor, and performs faceauthentication using the face information, which is included in theacquired information, of the visitor and registered face information,which has been registered in a member database, of each member.

RELATED DOCUMENT Patent Document

-   [Patent Document 1] Japanese Unexamined Patent Publication No.    2016-126749

SUMMARY OF THE INVENTION Technical Problem

In a technology for identifying a person to make settlement using a faceauthentication process, there is a problem in that accuracy ofidentification of the person to make settlement is lowered in a casewhere a plurality of persons are extracted from an image. An object ofthe present invention is to improve the accuracy of identification ofthe person to make settlement using the face authentication process.

According to the present invention, there is provided an informationprocessing system including:

-   -   an acquisition unit that acquires an image, which is generated        by a camera, of a face;    -   a selection unit that extracts a person in the acquired image,        and selects a person who performs a settlement process based on        at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification unit that identifies the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a settlement unit that performs the settlement process with        respect to the identified person.

In addition, according to the present invention, there is provided acustomer identification apparatus including:

-   -   an acquisition unit that acquires an image, which is generated        by a camera, of a face;    -   a selection unit that extracts a person in the acquired image,        and selects a person who performs a settlement process based on        at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification unit that identifies the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a notification unit that notifies information for identifying        the person who performs the settlement process and who is        identified by the identification unit to the checkout apparatus.

In addition, according to the present invention, there is provided aninformation processing method, which is performed by a computer, themethod including:

-   -   an acquisition step of acquiring an image, which is generated by        a camera, of a face;    -   a selection step of extracting a person in the acquired image,        and selecting a person who performs a settlement process based        on at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification step of identifying the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a settlement step of performing the settlement process with        respect to the identified person.

In addition, according to the present invention, there is provided aprogram causing a computer to function as:

-   -   an acquisition unit that acquires an image, which is generated        by a camera, of a face;    -   a selection unit that extracts a person in the acquired image,        and selects a person who performs a settlement process based on        at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification unit that identifies the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a settlement unit that performs the settlement process with        respect to the identified person.

Advantageous Effects of Invention

According to the present invention, it is possible to improve accuracyof identification of a person to make settlement using a faceauthentication process.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-described object, other objects, features, and advantages willbe further apparent with preferable example embodiments, which will bedescribed below, and the accompanying drawings below.

FIG. 1 is an example of a functional block diagram illustrating aninformation processing system of the present example embodiment.

FIG. 2 is a diagram conceptually illustrating an example of a hardwareconfiguration of a system and an apparatus of the present exampleembodiment.

FIG. 3 is a sequence diagram illustrating an example of a flow of aprocess of the information processing system of the present exampleembodiment.

FIG. 4 is a sequence diagram illustrating an example of the flow of theprocess of the information processing system of the present exampleembodiment.

FIG. 5 is a sequence diagram illustrating an example of the flow of theprocess of the information processing system of the present exampleembodiment.

FIG. 6 is an example of a functional block diagram of the informationprocessing system of the present example embodiment.

FIG. 7 is an example of a functional block diagram of a checkoutapparatus of the present example embodiment.

FIG. 8 is an example of a functional block diagram of a customeridentification apparatus of the present example embodiment.

FIG. 9 is a diagram illustrating a realization example of theinformation processing system of the present example embodiment.

FIG. 10 is a diagram illustrating a realization example of theinformation processing system of the present example embodiment.

FIG. 11 is an example of a functional block diagram of the informationprocessing system of the present example embodiment.

FIG. 12 is an example of a functional block diagram of the informationprocessing system of the present example embodiment.

FIG. 13 is an example of a functional block diagram of the informationprocessing system of the present example embodiment.

FIG. 14 is an example of a functional block diagram of the informationprocessing system of the present example embodiment.

DESCRIPTION OF EMBODIMENTS First Example Embodiment

FIG. 1 illustrates an example of a functional block diagram of aninformation processing system 100 of the present example embodiment. Asillustrated in the drawing, the information processing system 100includes an acquisition unit 11, an identification unit 12, a selectionunit 14, and a checkout unit 21. The information processing system 100of the present example embodiment is, for example, a point of sales(POS) register. It may be assumed that the POS register is operated by asales clerk or is operated by a customer.

Here, an example of a hardware configuration of the informationprocessing system 100 will be described. The respective functional unitsincluded in the information processing system 100 of the present exampleembodiment are realized by any combination of hardware and software ofany computer based on a Central Processing Unit (CPU), a memory, aprogram which is loaded on the memory, a storage unit (which is capableof storing a program which is downloaded from a storage medium, such asa Compact Disc (CD), a server on the Internet, or the like, in additionto a program that is previously stored from a stage in which anapparatus is delivered), such as a hard disk, which stores the program,and an interface for network connection. Furthermore, those skilled inthe art understand that various modification examples exist for a methodand an apparatus, which realize the store system.

FIG. 2 is a block diagram illustrating the hardware configuration of theinformation processing system 100 of the present example embodiment. Asillustrated in FIG. 2 , the information processing system 100 includes aprocessor 1A, a memory 2A, an input and output interface 3A, aperipheral circuit 4A, and a bus 5A. The peripheral circuit 4A includesvarious modules.

The bus 5A is a data transmission path which is used for the processor1A, the memory 2A, the peripheral circuit 4A, and the input and outputinterface 3A to transmit and receive data to and from each other. Theprocessor 1A is, for example, an arithmetic processing apparatus such asthe Central Processing Unit (CPU) or a Graphics Processing Unit (GPU).The memory 2A is, for example, a memory such as a Random Access Memory(RAM) or a Read Only Memory (ROM). The input and output interface 3Aincludes an interface which is used to acquire information from an inputapparatus (for example: a keyboard, a mouse, a microphone, a physicalkey, a touch panel display, a code reader, or the like), an externalapparatus, an external server, an external sensor, and the like, and aninterface which is used to output information to an output apparatus(for example: a display, a speaker, a printer, an emailer, or the like),the external apparatus, the external server, and the like. It ispossible for the processor 1A to output an instruction to each of themodules and to perform an arithmetic operation based on an arithmeticresult of each of the modules.

Subsequently, each functional unit illustrated in FIG. 1 will bedescribed in detail.

The checkout unit 21 includes a unit (registration unit) that performs aprocess (registration process) of registering a product to be checkedout, and a unit (a settlement unit) that performs a settlement process.

First, the registration process, which is performed by the checkout unit21, will be described. The checkout unit 21 includes a unit that is usedto input information for identifying the product to be checked out. Theinput is realized by various apparatuses such as the code reader, thephysical key, the touch panel display, the microphone, and the camera.

For example, the information for identifying the product may be input insuch a way that the code reader reads a code which is attached to theproduct and which indicates information for identifying the product. Inaddition, the information for identifying the product may be inputthrough operation of the physical key or the touch panel display. Inaddition, the information for identifying the product may be input byvoice through the microphone. In addition, the information foridentifying the product may be input using the camera that captures theproduct to be checked out and a unit that identifies a product includedin an image which is generated by the camera based on the feature valueof appearance of the product or a code (the information for identifyingthe product which is attached to the product) included in the mage.

It is possible for the checkout unit 21 to acquire product information(for example, price or the like) corresponding to the information foridentifying the product to be checked out from a previously preparedproduct master. Furthermore, it is possible for the checkout unit 21 tocompute the sum of prices of one or more products to be checked out.Furthermore, it is possible for the checkout unit 21 to display theinformation for identifying the product to be checked out, the computedsum, and the like on the display or the like.

Subsequently, a settlement process performed by the checkout unit 21will be described. The settlement process refers to completing a salestransaction in which the amount of money to be paid is decided, by aselected payment method (cash, a credit card, points, or the like). Itis possible for the checkout unit 21 to perform the settlement processusing electronic money information or point information, whichcorresponds to the customer identified through a face authenticationprocess. That is, in a case where the information for identifying thecustomer (a person who makes payment for the product) identified by theidentification unit 12, which will be described below, is acquired, itis possible for the checkout unit 21 to acquire the electronic moneyinformation or the point information, which corresponds to the customer,from the server. Furthermore, it is possible for the checkout unit 21 toperform the settlement process using the electronic money information orthe point information. That is, the settlement process of paying the sumof the prices of the products to be checked out using the electronicmoney or the points or a settlement process of adding points, which aregiven in present shopping, to remaining points indicated by the pointinformation may be performed. In addition, in a case of the informationprocessing system 100 in a store installed in an inside of anorganization such as a company, a settlement process may be performed byaccumulating the amount of money to be paid in association withinformation (for example: employee number) for identifying the customerwithin the organization, and deducting the amount of money to be paidfrom salaries.

It should be noted that, it may be possible for the checkout unit 21 toperform the settlement process using the cash, the credit card, anothertype of electronic money, another type of point, and the like. Forexample, in a case where the customer is not identified by theidentification unit 12, the settlement process may be performed usinganother method which is illustrated.

The checkout unit 21 performs the registration process and thesettlement process in this order. When an operation is performed tostart the settlement process after the registration process ofregistering at least one product, the checkout unit 21 starts thesettlement process according to the operation.

The operation performed to start the settlement process may be, forexample, pressing of the physical key corresponding to “start of thesettlement process”. In addition, the operation may be, for example, anoperation (for example: touch) to be performed on an area correspondingto the “start of the settlement process” over a touch panel area. Inaddition, the operation may be, for example, an input of a prescribedword corresponding to the “start of the settlement process” through themicrophone. Note that, the examples here are only examples, and thepresent invention is not limited thereto.

The acquisition unit 11 acquires the image (an image of a face of thecustomer) generated by the camera which captures the customer. Forexample, the camera is installed in a vicinity of the informationprocessing system 100, for example, installed to be integrated with theinformation processing system 100. Furthermore, the camera is installedto capture a customer who is located in the vicinity of the informationprocessing system 100, more specifically, a customer who is undercheckout. The camera may capture a moving image in succession or maycapture a still image at a prescribed timing. That is, the acquisitionunit 11 may acquire images corresponding to a plurality of frames whichare generated in succession at prescribed time intervals, or may acquireone image, which is generated at certain timing, from the camera.

It is possible for the selection unit 14 to extract a person in theimage acquired by the acquisition unit 11, and to select a person whoperforms the settlement process among a plurality of persons based on aat least one of a distance from the checkout apparatus (in a case of thepresent example embodiment, the information processing system 100), anappearance frequency (that is, a frequency acquired by capturing withthe camera during a prescribed time frame), and an appearance period oftime (that is, a period of time acquired by capturing with the cameraduring the prescribed time frame) in a case where the plurality ofpersons are extracted.

It is possible for the selection unit 14 of the present exampleembodiment to select the person who performs the settlement process fromamong the plurality of persons based on mainly the distance from thecheckout apparatus. In a case where the camera is installed in avicinity of the checkout apparatus, for example, installed to beintegrated with the checkout apparatus, a “distance between the cameraand the person” indicates a “distance between the checkout apparatus andthe person”.

For example, the selection unit 14 may select a person of which adistance from the camera is the shortest as the person who performs thesettlement process. In addition, the selection unit 14 may selectpersons of which the distance from the camera is equal to or less than areference value as candidates of the person who performs the settlementprocess. Furthermore, in a case where there exist a plurality ofcandidates who are selected based on the distance from the camera, theselection unit 14 may select the person who performs the settlementprocess from among the plurality of candidates based on anothercondition.

The distance from the camera may be estimated through image analysis.For example, a person who is photographed such that a length of aprescribed part (for example: an interval between eyes) of a body is thelongest may be determined as the person of which the distance from thecamera is the shortest. In addition, a person who is photographed suchthat the length of the prescribed part (for example: the intervalbetween eyes) of the body is equal to or larger than the reference valuemay be determined as the person of which the distance from the camera isequal to or less than the reference value. Note that, the distance fromthe camera may be estimated using another method.

Here, another condition, which is applied in a case where there existthe plurality of candidates who are selected based on the distance fromthe checkout apparatus, will be described. For example, the selectionunit 14 may select a person of which a period of time (a period of timeincluded in the image) acquired by capturing with the camera during theprescribed time frame is the longest as the person who performs thesettlement process from among the plurality of candidates. In addition,the selection unit 14 may select a person of which a frequency(frequency included in the image) acquired by capturing with the cameraduring the prescribed time frame is the highest as the person whoperforms the settlement process from among the plurality of candidates.The frequency acquired by capturing with the camera may be the number oftimes that is counted in such a way that, for example, extraction froman image corresponding to one frame is once. The period of time acquiredby capturing with the camera has the same meaning as a period of timeincluded in the moving image. In order to compute the period of time andfrequency acquired by capturing with the camera, it is necessary togroup the same person who is included over the images corresponding tothe plurality of frames. For example, the persons who are extracted fromthe image and of which feature values are similar may be collectivelygrouped.

A timing, in which the prescribed time frame starts, may be, forexample, a prescribed operation which is performed before an operationof starting the settlement process. Although a timing, in which aninitial product registration is performed, or the like is illustrated asan example, the present invention is not limited thereto. For example,although a timing, in which the operation of starting the settlementprocess is performed, or the like is illustrated as an example of atiming, in which the prescribed time frame ends, the present inventionis not limited thereto.

Another example of another condition will be described. For example, theselection unit 14 may select a person who performs a prescribedbehavior, which is previously decided, as the person who performs thesettlement process from among the plurality of candidates. Theprescribed behavior is a prescribed behavior which is considered to beperformed by the person who performs the settlement process. Forexample, although “raising hand”, “pointing themself using a finger orthe like”, and the like are exemplified, the present invention is notlimited thereto. These are behaviors which are considered to beperformed by the person who performs the settlement process in responseto an inquiry “who will pay for?” from the sales clerk.

Another example of another condition will be described. For example, theselection unit 14 may select a person who makes a remark, which ispreviously decided, as the person who performs the settlement processfrom among the plurality of candidates. Content of the remark isconsidered to be given by the person who performs the settlementprocess. For example, information for specifying a settlement method,and the like may be considered to be the content of the remark. That is,“cash”, a “card”, a name or a popular name of a prescribed paymentservice, and the like may be considered. For example, in a case where aname of the payment service using the face authentication is “facepass”, the selection unit 14 may identify a person who makes a remark of“face pass”.

In a case of the example, the information processing system 100 mayinclude a microphone (voice acquisition unit). Furthermore, theselection unit 14 may detect that a prescribed remark as described aboveis made by performing voice analysis on sounds which are collected bythe microphone. Furthermore, the selection unit 14 may analyze themoving image captured by the camera, and may identify a person whosemouth is moving (or a person who performs prescribed movementcorresponding to the content of the remark) at a timing in which theprescribed remark is detected.

It should be noted that, the selection unit 14 may select the person whoperforms the settlement process (a person who satisfies a condition ofthe distance from the camera or another condition) from among thepersons of which estimated ages estimated based on the image are equalto or larger than the reference value. In other words, the selectionunit 14 may exclude persons of which the estimated ages estimated basedon the image are less than the reference value from the person whoperforms the settlement process.

It should be noted that, in a case where persons of which the estimatedages are equal to or larger than the reference value and persons ofwhich the estimated ages are less than the reference value are extractedfrom the image, the selection unit 14 may select the person who performsthe settlement process from among the persons of which the estimatedages are equal to or larger than the reference value. That is, in a casewhere only the persons of which the estimated ages are less than thereference value are extracted, the selection unit 14 may select theperson who performs the settlement process from among the persons ofwhich the estimated ages are less than the reference value.

In addition, the selection unit 14 may select the person who performsthe settlement process (the person who satisfies the condition of thedistance from the camera and another condition) from among persons whomake movement. In other words, the selection unit 14 may exclude personswho do not make the movement from the person who performs the settlementprocess.

It should be noted that, in a case where one person is extracted fromthe image, it is possible for the selection unit 14 to select the personas the person who performs the settlement process.

The identification unit 12 identifies the person extracted in the imagebased on database information in which the feature value of anappearance of each of the plurality of persons is registered. Theidentification unit 12 extracts the feature value of the appearance ofthe person extracted from the image. Furthermore, the identificationunit 12 collates the feature value of the person extracted from theimage with the feature value, which is included in the databaseinformation, of the appearance of each of the plurality of persons, anddetermines a person of which a degree of similarity is equal to orlarger than the reference value.

In a case where it is possible to determine the person of which thedegree of similarity is equal to or larger than the reference value inthe database information, the identification unit 12 outputs informationfor identifying the person as an identification result. On the otherhand, in a case where it is not possible to determine the person ofwhich the degree of similarity is equal to or larger than the referencevalue in the database information, the identification unit 12 outputs“no person concerned” as the identification result.

Here, an example of a flow of a process of the information processingsystem 100 will be described with reference to a sequence diagram ofFIG. 3 . Here, it is assumed that the selection unit 14 selects a personof which a distance from the camera is the shortest as the person whoperforms the settlement process.

The information processing system 100 performs a registration andsettlement process and an authentication process (a process ofidentifying the person who performs the settlement process) in parallel.First, in a case where the product registration process starts in theregistration and settlement process (S10), a notification thereof isprovided to the authentication process (S11). According thereto, aprocess of identifying the customer starts in the authenticationprocess. Specifically, the image is acquired (S14), the person whoperforms the settlement process is selected from among the personsincluded in the image (S15). In S15, first, the persons included in theimage are extracted. Furthermore, in a case where the number of personsincluded in the image is plural, the person of which the distance fromthe camera is the shortest is selected as the person who performs thesettlement process. On the other hand, in a case where the number ofpersons included in the image is one, the one person is selected as theperson who performs the settlement process.

Thereafter, the information processing system 100 performs the processof identifying the person who performs the settlement process (S16).That is, the feature value of the appearance of the person who isselected to perform the settlement process in S15 is extracted, and acollation process is performed using the database information.Furthermore, the information processing system 100 accumulates theidentification results (information for identifying the person or noperson concerned) which are acquired through the collation process(S17). In the authentication process, S14 to S17 are repeated until anotification in S18 is received.

On the other hand, in the registration and settlement process, theproduct registration is received (S12) even after the productregistration process in S10 starts. While the operation of starting thesettlement process is not performed (No in S13), the productregistration is received (S12). In a case where the operation ofstarting the settlement process is performed (Yes in S13), anotification thereof is provided to the authentication process (S18).

In the authentication process, for example, the process of identifyingthe customer (S14 to S17) ends according to the notification of S18.Furthermore, from among the identification results (in which theinformation for identifying the person or no person concerned isassociated with each of the frames on which the image analysis isperformed) which are accumulated by that time, a prescribedidentification result (the information for identifying the person or noperson concerned, which is associated with a prescribed frame) isextracted (S19), and the prescribed identification result is notified tothe registration and settlement process (S20).

For example, the newest identification result at a point of time inwhich the notification in S18 is provided may be extracted. Furthermore,in a case where the newest identification result is the information foridentifying the person, the identification result may be notified to theregistration and settlement process. On the other hand, in a case wherethe newest identification result is no person concerned, anotheridentification result may be extracted under a prescribed searchcondition. Furthermore, in a case where another identification result isthe information for identifying the person, the identification resultmay be notified to the registration and settlement process. On the otherhand, in a case where another identification result is no personconcerned, another identification result may be further extracted undera prescribed search condition. Furthermore, in a case where theidentification result is no person concerned in succession as many as aprescribed number, no person concerned may be notified to theregistration and settlement process.

In the registration and settlement process, the settlement process isperformed based on the identification result acquired in S20 (S21). In acase where the information for identifying the person is acquired as theidentification result, for example, it is possible to acquire theelectronic money information or the point information, which correspondsto the information for identifying the person, from the server, and toperform the settlement process using the electronic money information orthe point information. On the other hand, in a case where no personconcerned is acquired as the identification result, it is possible toperform the settlement process using, for example, another method.

Here, an example in which the camera captures the moving image isdescribed. In a case where the camera captures the still image, theprocesses of S14 to S16 are performed according to the notification ofS11 in the authentication process, and the identification result isnotified to the registration and settlement process at any timing.Furthermore, in a case where the operation of starting the settlementprocess is received (Yes in S13) in the registration and settlementprocess, the settlement process is performed (S21) based on theidentification result which is notified from the authentication processat any timing.

Subsequently, another example of the flow of the process of theinformation processing system 100 of the present example embodiment willbe described with reference to a sequence diagram of FIG. 4 . Here, itis assumed that the selection unit 14 decides the person of which thedistance from the camera is equal to or less than the reference value asthe candidates of the person who performs the settlement process, anddecides a person who satisfies a prescribed condition as the person whoperforms the settlement process among candidates.

The information processing system 100 performs the registration andsettlement process and the authentication process (the process ofidentifying the person who performs the settlement process) in parallel.First, in a case where the product registration process starts in theregistration and settlement process (S30), a notification thereof isprovided to the authentication process (S31). In the authenticationprocess, the process of identifying the customer starts according to thenotification thereof. Specifically, the image is acquired (S34), and thecandidates who perform the settlement process are selected from amongthe persons included in the image (S35). In S35, first, the personsincluded in the image are extracted. Furthermore, the persons of whichthe distance from the camera is equal to or less than the referencevalue are selected as the candidates of the person who performs thesettlement process from among the persons included in the image.

Thereafter, the information processing system 100 performs a process ofidentifying each of the candidates of the person who performs thesettlement process (S36). That is, the feature value of the appearanceof each of the candidates is extracted, and the collation process isperformed using the database information. Furthermore, the informationprocessing system 100 accumulates the identification results(information for identifying the person or no person concerned) for eachof the candidates (S37). In the authentication process, S34 to S37 arerepeated until a notification of S38 is received.

On the other hand, in the registration and settlement process, theproduct registration is received (S32) even after the productregistration process in S30 starts. While the operation of starting thesettlement process is not performed (No in S33), the productregistration is received (S32). When the operation of starting thesettlement process is performed (Yes in S33), a notification thereof isprovided to the authentication process (S38).

In the authentication process, for example, the process of identifyingthe customer (S34 to S37) ends according to the notification of S38.Furthermore, the person who performs the settlement process is decidedamong the candidates of the person who performs the settlement process(S39), and an identification result of the person is notified to theregistration and settlement process (S40).

In S39, the information processing system 100 groups the same person whois selected as the candidate over the plurality of frames based on, forexample, results (for example: the identification result (informationfor identifying the person or no person concerned) of each of thecandidates and the feature value of the appearance are associated witheach frame on which the image analysis is performed) accumulated by thattime. For example, similar feature values of the respective personsextracted from the image may be collectively grouped. Furthermore, in acase where a plurality of groups are generated, the informationprocessing system 100 may decide that there exist a plurality ofcandidates of the person who performs the settlement process. On theother hand, in a case where one group is generated, the informationprocessing system 100 may decide that there exists one candidate of theperson who performs the settlement process.

In a case where there exist the plurality of candidates of the personwho performs the settlement process, it is possible to select a personwho satisfies another condition as the person who performs thesettlement process from among the candidates. Another condition is thesame as described above. On the other hand, in a case where there existsone candidate of the person who performs the settlement process, it ispossible to select the person as the person who performs the settlementprocess.

In the registration and settlement process, the settlement process isperformed based on the identification result acquired in S40 (S41). In acase where the information for identifying the person is acquired as theidentification result, for example, it is possible to acquire theelectronic money information or the point information, which correspondsto the information for identifying the person, from the server, and toperform the settlement process using the electronic money information orthe point information. On the other hand, in a case where no personconcerned is acquired as the identification result, it is possible toperform the settlement process using, for example, another method.

Here, the example in which the camera captures the moving image isdescribed. In the case where the camera captures the still image, S34,S35, S36, and S39 are performed in this order according to anotification of S31 in the authentication process, and theidentification result is notified to the registration and settlementprocess at any timing. Furthermore, in a case where the operation ofstarting the settlement process is received (Yes in S33) in theregistration and settlement process, the settlement process is performed(S41) based on the identification result which is notified from theauthentication process at any timing.

According to the above-described information processing system 100 ofthe present example embodiment, it is possible to select one person whoperforms the settlement process from among the plurality of persons, forexample, even in a case where the plurality of persons are included inthe image and the plurality of persons are extracted from the image. Asa result, it is possible to appropriately perform the settlement processbased on the information (for example: the electronic money informationor the point information) corresponding to the selected person.

In addition, it is possible for the information processing system 100 ofthe present example embodiment to select the person who performs thesettlement process based on the distance from the checkout apparatus.For example, it is considered that the person who performs thesettlement process is located near to the checkout apparatus in order toperform an operation for the settlement or to deal with the sales clerk.Therefore, it is possible for the information processing system 100 ofthe present example embodiment to select the person of which thedistance from the checkout apparatus is the shortest as the person whoperforms the settlement process. As a result, it is possible toaccurately select the person who performs the settlement process fromamong the plurality of extracted persons.

In addition, it is possible for the information processing system 100 toselect a person of which the distance from the checkout apparatus isequal to or less than the reference value and who satisfies anothercondition as the person who performs the settlement process. Inaddition, it is possible for the information processing system 100 toselect a person whose appearance frequency is equal to or larger than aprescribed value and who satisfies another condition as the person whoperforms the settlement process. In addition, it is possible for theinformation processing system 100 to select a person whose appearanceperiod of time is equal to or larger than a prescribed value and whosatisfies another condition as the person who performs the settlementprocess.

For example, it is considered that the person who performs thesettlement process is located in front of the information processingsystem 100 or the sales clerk for the longest time in order to performthe operation for the settlement or to deal with the sales clerk and isincluded in the image captured by the camera for the longest time.Therefore, it is possible for the information processing system 100 toselect the person of which the distance from the checkout apparatus isequal to or less than the reference value and of which the frequency orthe period of time acquired by capturing with the camera in theprescribed time frame (for example: time frame during which checkout isperformed) is the greatest as the person who performs the settlementprocess. As a result, it is possible to accurately select the person whoperforms the settlement process from among the plurality of extractedpersons.

In addition, it is possible to select the person of which the distancefrom the checkout apparatus is equal to or less than the reference valueand who performs prescribed speech and behavior for the settlement asthe person who performs the settlement process. For example, it ispossible to select a person who makes a remark for the payment method asthe person who performs the settlement process. As a result, it ispossible to accurately select the person who performs the settlementprocess from among the plurality of extracted persons.

In addition, for example, it is considered that a parent performssettlement in a case where the parent and a child come to a store.Therefore, for example, in a case where the person of which theestimated age is equal to or larger than the reference value and theperson of which the estimated age is less than the reference value areextracted, it is possible for the information processing system 100 ofthe present example embodiment to select the person who performs thesettlement process while excluding the person of which the estimated ageis less than the reference value. As a result, it is possible toaccurately select the person who performs the settlement process fromamong the plurality of extracted persons.

In addition, for example, in a case where the person who is included ina poster, a signboard, or the like is included in the image, theinformation processing system 100 may extract the person from the image.The information processing system 100 of the present example embodimentselects the person who performs the settlement process while excluding aperson who does not move, and thus it is possible to select the personwho performs the settlement process while excluding the person who isincluded in the poster, the signboard, or the like. As a result, it ispossible to accurately select the person who performs the settlementprocess from among the plurality of extracted persons.

In addition, in the present example embodiment, it is possible to selectthe person who performs the settlement process based on a condition,such as “the distance from the checkout apparatus is equal to or lessthan the reference value”, which can be relatively easily decided.Furthermore, in a case where the plurality of persons are selected withsuch an index, that is, only in a case where the persons are notnarrowed down to one person, it is possible to select the person whoperforms the settlement process based on another condition with a higherdegree of difficulty in detection. With the configuration, it ispossible to reduce a processing burden of the computer.

Second Example Embodiment

In an information processing system 100 of the present exampleembodiment, a configuration of the selection unit 14 is different fromthe first example embodiment. The other configurations are the same asin the first example embodiment.

In a case where persons included in the image are extracted and aplurality of persons are extracted, the selection unit 14 selects theperson who performs the settlement process based on mainly at least oneof the appearance frequency (that is, the frequency acquired bycapturing with the camera during the prescribed time frame) and theappearance period of time (that is, the period of time acquired bycapturing with the camera during the prescribed time frame).

For example, the selection unit 14 may select the person of which thefrequency acquired by capturing with the camera during the prescribedtime frame is the highest or the person of which the period of timeacquired by capturing with the camera during the prescribed time frameis the longest as the person who performs the settlement process. Inaddition, the selection unit 14 may select the person of which thefrequency acquired by capturing with the camera during the prescribedtime frame or of which the period of time acquired by capturing with thecamera during the prescribed time frame is equal to or larger than thereference value as the candidate of the person who performs thesettlement process. Furthermore, in a case where there exist theplurality of candidates who are selected based on the frequency acquiredby capturing with the camera during the prescribed time frame or theperiod of time acquired by capturing with the camera during theprescribed time frame, the selection unit 14 may select the person whoperforms the settlement process from among the plurality of candidatesbased on another condition.

Here, another condition will be described which is applied to the casewhere there exist the plurality of candidates who are selected based onthe frequency acquired by capturing with the camera during theprescribed time frame or the time acquired by capturing with the cameraduring the prescribed time frame. For example, the selection unit 14 mayselect the person of which the distance from the checkout apparatus (ina case of the present example embodiment, the information processingsystem 100) is the shortest as the person who performs the settlementprocess from among the plurality of candidates. For example, a person ofwhich a frequency that the distance from the checkout apparatus is thesmallest is high or of which a period of time that the distance from thecheckout apparatus is the smallest is long during the prescribed timeframe may be selected as the person who performs the settlement process.In addition, the person of which the distance from the checkoutapparatus is the shortest at a prescribed timing (for example: a timingin which the operation of starting the settlement process is performed)may be selected as the person who performs the settlement process.

Another example of another condition will be described. For example, theselection unit 14 may select the person who performs the prescribedbehavior, which is previously decided, as the person who performs thesettlement process from among the plurality of candidates. Theprescribed behavior is a prescribed behavior which is considered to beperformed by the person who performs the settlement process. Forexample, although “raising hand”, “pointing themself using a finger orthe like”, and the like are exemplified, the present invention is notlimited thereto. These are behaviors which are considered to beperformed by the person who performs the settlement process in responseto an inquiry “who will pay for?” from the sales clerk.

Another example of another condition will be described. For example, theselection unit 14 may select a person who makes a remark, which ispreviously decided, as the person who performs the settlement processfrom among the plurality of candidates. Content of the remark isconsidered to be given by the person who performs the settlementprocess. For example, information, which is used to specify a settlementmethod, and the like may be considered to be the content of the remark.That is, “cash”, a “card”, a name or a popular name of a prescribedpayment service, and the like may be considered. For example, in a casewhere a name of the payment service using the face authentication is“face pass”, the selection unit 14 may identify a person who makes aremark of “face pass”.

In a case of the example, the information processing system 100 mayinclude a microphone. Furthermore, the selection unit 14 may detect thata prescribed remark as described above is made by performing voiceanalysis on sounds which are collected by the microphone. Furthermore,the selection unit 14 may analyze a moving image captured by the camera,and may identify a person whose mouth is moving (or a person whoperforms prescribed movement corresponding to the content of the remark)at a timing in which the prescribed remark is detected.

It should be noted that, the selection unit 14 may select the person whoperforms the settlement process (a person who satisfies a condition ofthe distance from the camera or another condition) from among thepersons of which estimated ages estimated based on the image are equalto or larger than the reference value. In other words, the selectionunit 14 may exclude persons of which the estimated ages estimated basedon the image are less than the reference value from the person whoperforms the settlement process.

It should be noted that, in a case where the persons of which theestimated ages are equal to or larger than the reference value and thepersons of which the estimated ages are less than the reference valueare extracted from the image, the selection unit 14 may select theperson who performs the settlement process from among the persons ofwhich the estimated ages are equal to or larger than the referencevalue. That is, in a case where only the persons of which the estimatedages are less than the reference value are extracted, the selection unit14 may select the person who performs the settlement process from amongthe persons of which the estimated ages are less than the referencevalue.

In addition, the selection unit 14 may select the person who performsthe settlement process (the person who satisfies the condition of thedistance from the camera and another condition) from among persons whomake movement. In other words, the selection unit 14 may exclude personswho do not make the movement from the person who performs the settlementprocess.

It should be noted that, in a case where one person is extracted fromthe image, it is possible for the selection unit 14 to select the personas the person who performs the settlement process.

Here, an example of a flow of a process of the information processingsystem 100 of the present example embodiment will be described withreference to a sequence diagram of FIG. 5 . Here, the selection unit 14selects the person who performs the settlement process based on thefrequency acquired by capturing with the camera during the prescribedtime frame or the period of time acquired by capturing with the cameraduring the prescribed time frame. Note that, the camera captures themoving image.

The information processing system 100 performs a registration andsettlement process and an authentication process (a process ofidentifying the person who performs the settlement process) in parallel.First, in a case where the product registration process starts in theregistration and settlement process (S50), a notification thereof isprovided to the authentication process (S51).

In the registration and settlement process, the product registration isreceived even after the product registration process in S50 starts(S52). While the operation of starting the settlement process is notperformed (No in S53), the product registration is received (S52). In acase where the operation of starting the settlement process is performed(Yes in S53), a notification thereof is provided to the authenticationprocess (S54).

In the authentication process, the process of identifying the customerstarts according to the notification of S54. Specifically, the movingimage, which is generated by the camera during the prescribed time frame(for example: between the notification of S51 and the notification ofS54), is processed, and the person who performs the settlement processis selected (S55).

For example, the person of which the frequency acquired by capturingwith the camera during the prescribed time frame is the highest or theperson of which the period of time acquired by capturing with the cameraduring the prescribed time frame is the longest may be decided as theperson who performs the settlement process. In addition, the person ofwhich the frequency acquired by capturing with the camera during theprescribed time frame or of which the period of time acquired bycapturing with the camera during the prescribed time frame is equal toor larger than the reference value may be decided as the candidate ofthe person who performs the settlement process. Furthermore, in a casewhere there exist the plurality of candidates who are decided based onthe frequency acquired by capturing with the camera during theprescribed time frame or the period of time acquired by capturing withthe camera during the prescribed time frame, the person who performs thesettlement process may be selected from among the plurality ofcandidates based on another condition. The details are the same asdescribed above.

Thereafter, the feature value of the appearance of the person who isselected as the person who performs the settlement process is extractedfrom the image, and the person who performs the settlement process isidentified based on the feature value and the database information inwhich the feature value of the appearance of each of the plurality ofpersons is registered (S56). Furthermore, the identification result (theinformation for identifying the person or no person concerned) isnotified to the registration and settlement process (S57).

In the registration and settlement process, the settlement process isperformed based on the identification result acquired in S57 (S58). In acase where the information for identifying the person is acquired as theidentification result, for example, it is possible to acquire theelectronic money information or the point information, which correspondsto the information for identifying the person, from the server, toperform the settlement process using the electronic money information orthe point information. On the other hand, in a case where no personconcerned is acquired as the identification result, it is possible toperform the settlement process using for example, another method.

According to the above-described information processing system 100 ofthe present example embodiment, for example, even in a case where theplurality of persons are included in the image and the plurality ofpersons are extracted from the image, it is possible to select oneperson who performs the settlement process from among the plurality ofpersons. As a result, it is possible to appropriately perform thesettlement process based on the information (for example: the electronicmoney information or the point information) corresponding to theselected person.

In addition, it is possible for the information processing system 100 ofthe present example embodiment to select the person who performs thesettlement process based on at least one of the frequency acquired bycapturing with the camera during the prescribed time frame and theperiod of time acquired by capturing with the camera during theprescribed time frame. For example, it is considered that the person whoperforms the settlement process is located in front of the informationprocessing system 100 or the sales clerk for the longest time in orderto perform the operation for the settlement or to deal with the salesclerk and is included in the image captured by the camera for thelongest time. Therefore, it is possible for the information processingsystem 100 to select the person of which the frequency acquired bycapturing with the camera in the prescribed time frame (for example:time frame during which checkout is performed) is the highest or ofwhich the period of time acquired by capturing with the camera is thelongest as the person who performs the settlement process. As a result,it is possible to accurately select the person who performs thesettlement process from among the plurality of extracted persons.

In addition, it is possible for the information processing system 100 toselect the person of which the frequency acquired by capturing with thecamera during the prescribed time frame or of which the period of timeacquired by capturing with the camera during the prescribed time frameis equal to or larger than the reference value and who satisfies anothercondition as the person who performs the settlement process.

For example, it is considered that the person who performs thesettlement process is located near to the information processing system100 in order to perform the operation for the settlement or to deal withthe sales clerk. Therefore, it is possible for the informationprocessing system 100 of the present example embodiment to decide theperson of which the frequency acquired by capturing with the cameraduring the prescribed time frame or of which the period of time acquiredby capturing with the camera during the prescribed time frame is equalto or larger than the reference value and of which the distance from theinformation processing system 100 is the shortest as the person whoperforms the settlement process. As a result, it is possible toaccurately select the person who performs the settlement process fromamong the plurality of extracted persons.

In addition, it is possible to select the person of which the frequencyacquired by capturing with the camera during the prescribed time frameor of which the period of time acquired by capturing with the cameraduring the prescribed time frame is equal to or larger than thereference value and who performs prescribed speech and behavior for thesettlement as the person who performs the settlement process. Forexample, it is possible to select a person who makes a remark about apayment method as the person who performs the settlement process. As aresult, it is possible to accurately select the person who performs thesettlement process from among the plurality of extracted persons.

In addition, for example, it is considered that a parent performssettlement in a case where the parent and a child come to a store.Therefore, for example, in a case where the person of which theestimated age is equal to or larger than the reference value and theperson of which the estimated age is less than the reference value areextracted, it is possible for the information processing system 100 ofthe present example embodiment to select the person who performs thesettlement process while excluding the person of which the estimated ageis less than the reference value. As a result, it is possible toaccurately select the person who performs the settlement process fromamong the plurality of extracted persons.

In addition, for example, in a case where the person who is included ina poster, a signboard, or the like is included in the image, theinformation processing system 100 may extract the person from the image.The information processing system 100 of the present example embodimentselects the person who performs the settlement process while excluding aperson who does not move, and thus it is possible to select the personwho performs the settlement process while excluding the person who isincluded in the poster, the signboard, or the like. As a result, it ispossible to accurately select the person who performs the settlementprocess from among the plurality of extracted persons.

In addition, in the present example embodiment, it is possible to selectthe person who performs the settlement process based on a condition,such as “the frequency acquired by capturing with the camera during theprescribed time frame or the period of time acquired by capturing withthe camera during the prescribed time frame is equal to or larger thanthe reference value”, which can be relatively easily decided.Furthermore, in a case where the plurality of persons are selected withsuch an index, that is, only in a case where the persons are notnarrowed down to one person, it is possible to select the person whoperforms the settlement process based on another condition with a higherdegree of difficulty in detection. With the configuration, it ispossible to reduce a processing burden of the computer.

Third Example Embodiment

FIG. 11 illustrates an example of a functional block diagram of aninformation processing system 100 of the present example embodiment. Theinformation processing system 100 of the present example embodiment isdifferent from those of the first and second example embodiments in thatthe information processing system 100 includes a second selection unit15. The other configurations are the same as in the first and secondexample embodiments.

The second selection unit 15 selects a person who performs a prescribedbehavior as a person who performs a settlement process.

For example, the second selection unit 15 may select the person whoperforms the prescribed behavior, which is previously decided, as theperson who performs the settlement process from among persons includedin the image acquired by the acquisition unit 11. The prescribedbehavior is a prescribed behavior which is considered to be performed bythe person who performs the settlement process. For example, although“raising hand”, “pointing themself using a finger or the like”, and thelike are exemplified, the present invention is not limited thereto.These are behaviors which are considered to be performed by the personwho performs the settlement process in response to an inquiry “who willpay for?” from the sales clerk.

It should be noted that, in a case where a plurality of persons areselected by the second selection unit 15, a process of selecting theperson who performs the settlement process from among the plurality ofpersons may be performed by the selection unit 14. For example, it ispossible for the selection unit 14 to select a person of which adistance from a checkout apparatus is the shortest, a person of whichthe frequency acquired by capturing with a camera during a prescribedtime frame is the highest, or a person of which the period of timeacquired by capturing with the camera is long, and the like as theperson who performs the settlement process among the plurality ofpersons selected by the second selection unit 15.

In the present example embodiment, it is also possible to realize thesame advantageous effects as in the first and second exampleembodiments. In addition, according to the present example embodiment,it is possible to perform a process of the selection unit 14 aftercandidates of the person who performs the settlement process beingnarrowed down through a process of the second selection unit 15.

Fourth Example Embodiment

FIG. 12 illustrates an example of a functional block diagram of aninformation processing system 100 of the present example embodiment. Theinformation processing system 100 of the present example embodiment isdifferent from those of the first and second example embodiments in thatthe information processing system 100 includes a third selection unit16. The other configurations are the same as in the first and secondexample embodiments.

The third selection unit 16 selects a person who makes a prescribedremark as a person who performs a settlement process.

For example, the third selection unit 16 may select a person who makes aremark, which is previously device, as the person who performs thesettlement process among persons included in the image acquired by theacquisition unit 11. Content of the remark is considered to be given bythe person who performs the settlement process. For example,information, which is used to identify a settlement method, and the likemay be considered to be the content of the remark. That is, “cash”, a“card”, a name or a popular name of a prescribed payment service, andthe like may be considered. For example, in a case where the name of thepayment service using a face authentication is “face pass”, the secondselection unit 15 may identify the person who makes the remark of “facepass”.

In a case of the example, the information processing system 100 mayinclude a microphone (voice acquisition unit). Furthermore, in a casewhere voice analysis is performed on sounds which are collected by themicrophone, the third selection unit 16 may detect that the prescribedremark, which is as described above, is made. Furthermore, the thirdselection unit 16 may analyze a moving image captured by a camera, andmay identify a person whose mouth is moving (or a person who isperforming prescribed movement corresponding to the content of theremark) at a timing in which the prescribed remark is detected.

It should be noted that, in a case where a plurality of persons areselected by the third selection unit 16, a process of selecting theperson who performs the settlement process from among the plurality ofpersons may be performed by the selection unit 14. For example, it ispossible for the selection unit 14 to select a person of which adistance from the checkout apparatus is the shortest, a person of whichthe frequency acquired by capturing with the camera during a prescribedtime frame is the highest, or a person of which the period of timeacquired by capturing with the camera is long, and the like as theperson who performs the settlement process among the plurality ofpersons selected by the third selection unit 16.

In the present example embodiment, it is also possible to realize thesame advantageous effects as in the first and second exampleembodiments. In addition, according to the present example embodiment,it is possible to perform a process of the selection unit 14 aftercandidates of the person who performs the settlement process beingnarrowed down through a process of the third selection unit 16.

Fifth Example Embodiment

FIG. 13 is an example of a functional block diagram of an informationprocessing system 100 of the present example embodiment. The informationprocessing system 100 of the present example embodiment is differentfrom those of the first and second example embodiments in that theinformation processing system 100 includes a fourth selection unit 17.The other configurations are the same as in the first and second exampleembodiments.

The fourth selection unit 17 selects a person of which an estimated ageestimated based on an image acquired by an acquisition unit 11 is equalto or larger than a reference value as a person who performs thesettlement process.

In a case where the person of which the estimated age is equal to orlarger than the reference value and the persons of which the estimatedages are less than the reference value are extracted from the image, thefourth selection unit 17 may select the person of which the estimatedage is equal to or larger than the reference value as the person whoperforms the settlement process. Furthermore, in a case where only theperson of which the estimated age is less than the reference value isextracted, the fourth selection unit 17 may select the person of whichthe estimated age is less than the reference value as the person whoperforms the settlement process.

It should be noted that, in a case where a plurality of persons areselected by the fourth selection unit 17, a process of selecting theperson who performs the settlement process from among the plurality ofpersons may be performed by the selection unit 14. For example, it ispossible for the selection unit 14 to select a person of which adistance from a checkout apparatus is the shortest, a person of whichthe frequency acquired by capturing with a camera during a prescribedtime frame is the highest, or a person of which the period of timeacquired by capturing with the camera is long, and the like as theperson who performs the settlement process among the plurality ofpersons selected by the fourth selection unit 17.

In the present example embodiment, it is also possible to realize thesame advantageous effects as in the first and second exampleembodiments. In addition, according to the present example embodiment,it is possible to perform a process of the selection unit 14 aftercandidates of the person who performs the settlement process beingnarrowed down through a process of the fourth selection unit 17.

Sixth Example Embodiment

FIG. 14 is an example of a functional block diagram of an informationprocessing system 100 of the present example embodiment. The informationprocessing system 100 of the present example embodiment is differentfrom those of the first and second example embodiments in that theinformation processing system 100 includes a fifth selection unit 18.The other configurations are the same as in the first and second exampleembodiments.

The fifth selection unit 18 selects a person who makes movement as aperson who performs the settlement process.

It should be noted that, in a case where a plurality of persons areselected by the fifth selection unit 18, a process of selecting theperson who performs the settlement process from among the plurality ofpersons may be performed by the selection unit 14. For example, it ispossible for the selection unit 14 to select a person of which adistance from a checkout apparatus is the shortest, a person of whichthe frequency acquired by capturing with a camera during the prescribedtime frame is the highest, or a person of which the period of timeacquired by capturing with the camera is long, and the like as theperson who performs the settlement process among the plurality ofpersons selected by the fifth selection unit 18.

In the present example embodiment, it is also possible to realize thesame advantageous effects as in the first and second exampleembodiments. In addition, according to the present example embodiment,it is possible to perform a process of the selection unit 14 aftercandidates of the person who performs the settlement process beingnarrowed down through a process of the fifth selection unit 18.

Seventh Example Embodiment

As illustrated in FIG. 6 , an information processing system 100 of thepresent example embodiment includes a customer identification apparatus10 and a checkout apparatus 20. An information processing system 100realizes the prescribed process described in the first to sixth exampleembodiments through a process cooperatively performed by the customeridentification apparatus 10 and the checkout apparatus 20. Although arealization example will be described later, the customer identificationapparatus 10 and the checkout apparatus 20 may be configured to bephysically and/or logically integrated, or may be configured to bephysically and/or logically separated.

First, an example of a hardware configuration of the customeridentification apparatus 10 and the checkout apparatus 20 will bedescribed. The respective functional units, which are included in thecustomer identification apparatus 10 and the checkout apparatus 20 ofthe present example embodiment, are realized by any combination ofhardware and software of any computer based on a CPU, a memory, aprogram which is loaded on the memory, a storage unit (which is capableof storing a program which is downloaded from a storage medium, such asCD, a server on the Internet, or the like, in addition to a program thatis previously stored from a stage in which an apparatus is delivered),such as a hard disk, which stores the program, and an interface fornetwork connection. Furthermore, those skilled in the art understandthat various modification examples exist for a realization method and anapparatus thereof.

FIG. 2 is a block diagram illustrating the hardware configuration of thecustomer identification apparatus 10 and the checkout apparatus 20 ofthe present example embodiment. As illustrated in FIG. 2 , the customeridentification apparatus 10 and the checkout apparatus 20 include theprocessor 1A, the memory 2A, the input and output interface 3A, theperipheral circuit 4A, and the bus 5A. The peripheral circuit 4Aincludes various modules. Note that, in a case where the customeridentification apparatus 10 and the checkout apparatus 20 are configuredto be physically and/or logically separated, each of the customeridentification apparatus 10 and the checkout apparatus 20 includes thehardware configuration as illustrated in FIG. 2 .

The bus 5A is a data transmission path which is used for the processor1A, the memory 2A, the peripheral circuit 4A, and the input and outputinterface 3A to transmit and receive data to and from each other. Theprocessor 1A is, for example, an arithmetic processing unit such as theCentral Processing Unit (CPU) or a Graphics Processing Unit (GPU). Thememory 2A is, for example, a memory such as a Random Access Memory (RAM)or a Read Only Memory (ROM). The input and output interface 3A includesan interface which is used to acquire information from an inputapparatus (for example: a keyboard, a mouse, a microphone, a physicalkey, a touch panel display, a code reader, or the like), an externalapparatus, an external server, an external sensor, and the like, and aninterface which is used to output information to an output apparatus(for example: a display, a speaker, a printer, an emailer, or the like),the external apparatus, the external server, and the like. It ispossible for the processor 1A to output an instruction to each of themodules and to perform an arithmetic operation based on an arithmeticresult of each of the modules.

FIG. 7 illustrates an example of a functional block diagram of thecheckout apparatus 20. As illustrated in the drawing, the checkoutapparatus 20 includes a checkout unit 21. Since a configuration of thecheckout unit 21 is the same as in the first to sixth exampleembodiments, the description thereof will not be repeated.

FIG. 8 illustrates an example of a functional block diagram of thecustomer identification apparatus 10. As illustrated in the drawing, thecustomer identification apparatus 10 includes an acquisition unit 11, anidentification unit 12, a notification unit 13, and a selection unit 14.The customer identification apparatus 10 may further include a secondselection unit 15, a third selection unit 16, a fourth selection unit17, or a fifth selection unit 18. Since configurations of theacquisition unit 11, the identification unit 12, the selection unit 14,the second selection unit 15, the third selection unit 16, the fourthselection unit 17, and the fifth selection unit 18 are the same as inthe first to sixth example embodiments, the description thereof will notbe repeated.

The notification unit 13 notifies an identification result of theidentification unit 12 with respect to a person who is decided as aperson who performs a settlement process by the selection unit 14 to thecheckout apparatus 20. The identification result is information foridentifying the person who performs the settlement process or no personconcerned.

It should be noted that, it is possible for the customer identificationapparatus 10 and the checkout apparatus 20 to transmit and receiveinformation to and from each other. For example, the checkout apparatus20 acquires the identification result from the customer identificationapparatus 10.

In addition, the checkout apparatus 20 notifies that a prescribedoperation is performed to the customer identification apparatus 10. Asan example, when the prescribed operation, which is performed before theoperation of starting the settlement process is performed, is performedduring a checkout process (a registration process, the settlementprocess, or the like) with respect to a certain customer, the checkoutapparatus 20 may notify that the operation is performed to the customeridentification apparatus 10 in response to the operation. For example,the checkout apparatus 20 may notify that an initial productregistration is performed during the checkout process with respect tothe customer to the customer identification apparatus 10. As anotherexample, the checkout apparatus 20 may notify that the operation ofstarting the settlement process is performed to the customeridentification apparatus 10.

A flow of a process of the information processing system 100 of thepresent example embodiment is the same as those illustrated in thesequence diagrams of FIGS. 3 and 5 . The registration and settlementprocess illustrated in the drawing is performed by the checkoutapparatus 20, and the authentication process illustrated in the drawingis performed by the customer identification apparatus 10.

Subsequently, a realization example of the information processing system100 of the present example embodiment, which is realized by the customeridentification apparatus 10 and the checkout apparatus 20, will bedescribed.

As illustrated in FIG. 9 , the customer identification apparatus 10 andthe checkout apparatus 20 may be realized by a POS register 1 and anattachment apparatus 2 which are physically and/or logically separated.The POS register 1 and the attachment apparatus 2 are configured to beable to communicate with each other in wired and/or wireless manner. ThePOS register 1 and the attachment apparatus 2 may be connected one toone as illustrated in the drawing, or a plurality of POS registers 1 maybe connected to one attachment apparatus 2.

In a case of an example of FIG. 9 , the checkout apparatus 20 may berealized by the POS register 1 and the customer identification apparatus10 may be realized by the attachment apparatus 2.

In addition, the checkout apparatus 20 and a partial function of thecustomer identification apparatus 10 may be realized by the POS register1, and another partial function of the customer identification apparatus10 may be realized by the attachment apparatus 2. For example, theacquisition unit 11 of the customer identification apparatus 10 and thepartial function (for example: a function of extracting a person from animage and a function of extracting a feature value of the person) of theidentification unit 12 may be realized by the POS register 1.Furthermore, another partial function (for example: a function ofcollating the feature value of the person extracted from the image withdatabase information in which a feature value of appearance of each ofthe plurality of persons is registered) of the identification unit 12,the notification unit 13, and the selection unit 14 may be realized bythe attachment apparatus 2. The attachment apparatus 2 may furtherinclude the second selection unit 15, the third selection unit 16, thefourth selection unit 17, or the fifth selection unit 18.

According to the above-described information processing system 100 ofthe present example embodiment, advantageous effects which are the sameas in the first and second example embodiments are acquired. Inaddition, as illustrated in FIG. 9 , with a configuration in which theinformation processing system 100 is realized by the POS register 1 andanother apparatus (attachment apparatus 2), it is possible to use anexisting POS register 1 which does not include a function of executingthe face authentication process. As a result, it is possible to savecosts for introducing a new system.

Eighth Example Embodiment

In the present example embodiment, an information processing system 100is realized by a customer identification apparatus 10 and a checkoutapparatus 20 as the same as in the seventh example embodiment. Arealization example of the information processing system 100 of thepresent example embodiment is different from that of the seventh exampleembodiment. The other configurations are the same as in the seventhexample embodiment.

In the present example embodiment, as illustrated in FIG. 10 , thecustomer identification apparatus 10 and the checkout apparatus 20 arerealized by a POS register 1 and a server 3 which are physically and/orlogically separated. The POS register 1 and the server 3 are configuredto be able to communicate with each other in wired and/or wirelessmanner. As illustrated in the drawing, a plurality of POS registers 1are connected to one server 3.

In a case of an example of FIG. 10 , the checkout apparatus 20 may berealized by the POS register 1 and the customer identification apparatus10 may be realized by the server 3.

In addition, the checkout apparatus 20 and a partial function of thecustomer identification apparatus 10 may be realized by the POS register1, and another partial function of the customer identification apparatus10 may be realized by the server 3. For example, an acquisition unit 11of the customer identification apparatus 10 and a partial function (forexample: a function of extracting a person from an image and a functionof extracting a feature value of the person) of an identification unit12 may be realized by the POS register 1. Furthermore, another partialfunction (for example: a function of collating the feature value of theperson extracted from the image with database information in which afeature value of appearance of each of the plurality of persons isregistered) of the identification unit 12, a notification unit 13, and aselection unit 14 may be realized by the server 3. The server 3 mayfurther include a second selection unit 15, a third selection unit 16, afourth selection unit 17, or a fifth selection unit 18.

According to the above-described information processing system 100 ofthe present example embodiment, advantageous effects which are the sameas in the first to seventh example embodiments are acquired. Inaddition, as illustrated in FIG. 10 , with a configuration in which theinformation processing system 100 is realized by the POS register 1 andanother apparatus (server 3), it is possible to use an existing POSregister 1 which does not include a function of performing a faceauthentication process. As a result, it is possible to save costs forintroducing a new system.

Although a part or whole of the above example embodiments can bedescribed as appendixes below, the present invention is not limited tobelow.

-   -   1. An information processing system including:    -   an acquisition unit that acquires an image, which is generated        by a camera, of a face;    -   a selection unit that extracts a person in the acquired image,        and selects a person who performs a settlement process based on        at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification unit that identifies the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a settlement unit that performs the settlement process with        respect to the identified person.    -   2. The information processing system of 1,    -   in which the selection unit decides a person of which a distance        from the checkout apparatus is short as the person who performs        the settlement process.    -   3. The information processing system of 1,    -   in which the selection unit selects at least any one of a person        of which a frequency acquired by capturing with the camera        during a prescribed time frame is high and a person of which a        period of time acquired by capturing with the camera during a        prescribed time frame is long as the person who performs the        settlement process.    -   4. The information processing system of any one of 1 to 3,        further including:    -   a second selection unit that selects a person who performs a        prescribed behavior as the person who performs the settlement        process.    -   5. The information processing system of any one of 1 to 4,        further including:    -   a voice acquisition unit that acquires a voice; and    -   a third selection unit that selects a person who makes a        prescribed remark as the person who performs the settlement        process based on the acquired voice.    -   6. The information processing system of any one of 1 to 5,        further including:    -   a fourth selection unit that selects a person of which estimated        age estimated based on the image is equal to or larger than a        reference value as the person who performs the settlement        process.    -   7. The information processing system of any one of 1 to 6,        further including:    -   a fifth selection unit that selects a person who makes movement        as the person who performs the settlement process.    -   8. A customer identification apparatus including:    -   an acquisition unit that acquires an image, which is generated        by a camera, of a face;    -   a selection unit that extracts a person in the acquired image,        and selects a person who performs a settlement process based on        at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification unit that identifies the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a notification unit that notifies information for identifying        the person with respect to the person who performs the        settlement process and who is identified by the identification        unit to the checkout apparatus.    -   9. An information processing method, which is performed by a        computer, the method including:    -   an acquisition step of acquiring an image, which is generated by        a camera, of a face;    -   a selection step of extracting a person in the acquired image,        and selecting a person who performs a settlement process based        on at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification step of identifying the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a settlement step of performing the settlement process with        respect to the identified person.    -   10. A program causing a computer to function as:    -   an acquisition unit that acquires an image, which is generated        by a camera, of a face;    -   a selection unit that extracts a person in the acquired image,        and selects a person who performs a settlement process based on        at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification unit that identifies the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a settlement unit that performs the settlement process with        respect to the identified person.    -   11. A customer identification method, which is performed by a        computer, the method including:    -   an acquisition step of acquiring an image, which is generated by        a camera, of a face;    -   a selection step of extracting a person in the acquired image,        and selecting a person who performs a settlement process based        on at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification step of identifying the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a notification step of notifying information for identifying the        person with respect to the person who performs the settlement        process and who is identified by the identification unit to the        checkout apparatus.    -   12. A program causing a computer to function as:    -   an acquisition unit that acquires an image, which is generated        by a camera, of a face;    -   a selection unit that extracts a person in the acquired image,        and selects a person who performs a settlement process based on        at least one of a distance from a checkout apparatus, an        appearance frequency, and an appearance period of time in a case        where a plurality of persons are extracted;    -   an identification unit that identifies the selected person based        on the image in which the person is included, and database        information in which a feature value of a face is registered;        and    -   a notification unit that notifies information for identifying        the person with respect to the person who performs the        settlement process and who is identified by the identification        unit to the checkout apparatus.

Hereinabove, although the present invention is described with referenceto the example embodiments, the present invention is not limited to theabove example embodiments. In the configurations and details of thepresent invention, various modifications which can be understood bythose skilled in the art in a scope of the present invention.

This application claims priority based on Japanese Patent ApplicationNo. 2016-248811 filed on Dec. 22, 2016, and the content of theapplication is incorporated herein by reference in its entirety.

The invention claimed is:
 1. An information processing systemcomprising: at least one memory configured to store instructions; and atleast one processor configured to execute the instructions to: acquirean image generated by a camera; extract a person in the acquired image;select a person who performs a settlement process based on an appearanceperiod of time in a case where a plurality of persons are extracted;identify the selected person based on the image in which the person isincluded, and database information in which a feature value of a face isregistered; perform the settlement process with respect to theidentified person; select a person of which the appearance period oftime is equal to or larger than a reference value, or a person of whichthe appearance period of time is the largest.
 2. The informationprocessing system according to claim 1, wherein the processor is furtherconfigured to execute the instructions to select, in a case where aplurality of persons are selected based on the appearance period oftime, a person of which frequency captured by the camera during aprescribed time is the highest or a person of which a distance from acheckout apparatus is the shortest.
 3. The information processing systemaccording to claim 1, wherein the processor is further configured toexecute the instructions to select, in a case where a plurality ofpersons are selected based on the appearance period of time, a personwho performs a prescribed behavior.
 4. The information processing systemaccording to claim 1, wherein the processor is further configured toexecute the instructions to: acquire a voice, and select, in a casewhere a plurality of persons are selected based on the appearance periodof time, a person who makes a prescribed remark.
 5. The informationprocessing system according to claim 1, wherein the processor is furtherconfigured to execute the instructions to select, in a case where aplurality of persons are selected based on the appearance period oftime, a person of which estimated age estimated based on the image isequal to or larger than a reference value.
 6. The information processingsystem according to claim 1, wherein the processor is further configuredto execute the instructions to select, in a case where a plurality ofpersons are selected based on the appearance period of time, a personwho makes movement.
 7. The information processing system according toclaim 1, wherein the processor is further configured to execute theinstructions to exclude, in a case where a plurality of persons areselected based on the appearance period of time, a person of whichestimated age estimated based on the image is less than a referencevalue.
 8. The information processing system according to claim 1,wherein the processor is further configured to execute the instructionsto exclude, in a case where a plurality of persons are selected based onthe appearance period of time, a person who does not make movement. 9.An information processing method, which is performed by a computer, themethod comprising: acquiring an image generated by a camera; extractinga person in the acquired image; selecting a person who performs asettlement process based on an appearance period of time in a case wherea plurality of persons are extracted; identifying the selected personbased on the image in which the person is included, and databaseinformation in which a feature value of a face is registered; performingthe settlement process with respect to the identified person; andselecting a person of which the appearance period of time is equal to orlarger than a reference value, or a person of which the appearanceperiod of time is the largest.
 10. A non-transitory storage mediumstoring a program causing a computer to: acquire an image generated by acamera; extract a person in the acquired image; select a person whoperforms a settlement process based on an appearance period of time in acase where a plurality of persons are extracted; identify the selectedperson based on the image in which the person is included, and databaseinformation in which a feature value of a face is registered; performthe settlement process with respect to the identified person; and selecta person of which the appearance period of time is equal to or largerthan a reference value, or a person of which the appearance period oftime is the largest.